Papers by Sujan Reddy A
Automating Human Evaluation of Dialogue Systems (2022.naacl-srw)
Copied to clipboard
| Challenge: | a recent study shows that human evaluations of dialogue systems weakly reflect human judgments. |
| Approach: | They propose a BERT-based model that fine-tunes a model with three prediction heads to predict whether the system-generated output is natural, fluent, and informative. |
| Outcome: | The proposed model achieves an average accuracy of 77% over the 3 labels . it also uses three different models to compute the labels compared to three separate models . |